Data path evaluation system and method

ABSTRACT

A data path evaluation system and method is described wherein the data path can be a voice signal path and the reference signal can contain white gaussian noise.

BACKGROUND

[0001] This embodiment of the invention relates to evaluation of datapaths in devices, and more particularly to evaluation of voice datapaths in telephony network devices.

[0002] As new hardware designs are implemented, both in data processingnetworks in general and in telephony voice processing networks inparticular, it is important to be able to validate the reliability andintegrity of the data path. In digital telephone emulation systems asignificant portion of the hardware front-end is dedicated to handlingthe voice data path. With the high complexity of such hardware, thelikelihood of loss of signal integrity is increased. Generally, alistening test does not provide enough information to validate thereliability and integrity of this path. Errors in this processing may goundetected by typical subjective audio quality tests. In addition, alistening test rarely provides information to what the source of aproblem may be.

[0003] Therefore, it would be highly desirable to quantitativelyevaluate the hardware design in a manner which enables identification ofthe source of the problems.

BRIEF DESCRIPTION OF THE DRAWING FIGURES

[0004]FIG. 1 is a block diagram of a general form of a data pathevaluation system according to one embodiment of the invention;

[0005]FIG. 2 is a block diagram of a data path evaluation systemaccording to one embodiment of the invention;

[0006]FIGS. 3 and 4 are block diagrams illustrating an alternative formof a data path evaluation system according to one embodiment of theinvention;

[0007]FIG. 5 is a block diagram illustrating another alternative form ofa data path evaluation system according to one embodiment of theinvention;

[0008]FIG. 6 is a block diagram illustrating a data path evaluationmethod according to one embodiment of the invention;

[0009]FIG. 7 is a graph including waveforms further illustrating thealignment stage in a method according to one embodiment of theinvention; and

[0010]FIGS. 8A and 8B are schematic block diagrams further illustratinga filter compensation stage in a method according to one embodiment ofthe invention.

DETAILED DESCRIPTION

[0011]FIG. 1 shows a general form of the data path evaluation systemaccording to the invention. A device under test 10 (DUT) has a data pathor channel 12 therethrough which may have two ends. Device 10 also mayhave a plurality of ports, two of which are designated 14 and 16 inFIG. 1. The system of FIG. 1 also includes a device 20 to provide asignal path of the type to which the DUT 10 will be operatively coupledduring use. In a telephony application, device 20 can comprise atelephone switch such as a private branch exchange (PBX). In a telephonyemulation application, device under test 10 is the front end hardwarehaving a PBX interface portion 24 and a system interface portion 26.Port 16 of DUT 10 may be connected via a signal path 30 to port 32 ofdevice 20. Device 20, of course, may have other forms such as theInternet or any device providing an appropriate signal transmission pathor channel.

[0012] The data path evaluation system of FIG. 1 further comprises ahost arrangement to provide a test signal and to define play and recordpaths. In the system shown, the host arrangement may include thecombination of a device under test host (DUT Host) 40, a test or captivehost 42 and a test or captive device 44. The DUT Host 40 may be apersonal computer or an embedded processor. The test host 42 in thesystem shown may be a separate host, although as will be shown presentlyit may be the DUT Host. The test or captive device 44 may be the samedevice as DUT 10, an additional device similar to DUT 10 or anotherdevice that interfaces to PBX 20 and that can capture voice signals andput voice signal patterns on the line. An example of the latter is thevoice product commercially available from the Voice Technologies Groupdivision of Dialogic Corporation under the trademark iPOD. In a manualrather than automated approach, host 40 and device 44 could even be thecombination of a scope and a signal generator and vice-versa. The DUTHost 40 may be connected to port 14 of DUT 10. Test host 42 and capturedevice 44 may be operatively connected as shown, and capture device 44may be connected to port 48 of PBX 20.

[0013] The data path evaluation system and method evaluates a deviceunder test based on captured data at both the input and output of thedevice while the device is not in normal service. This data is capturedfor two cases in order to test both the transmit hardware and thereceive hardware. The evaluation applies signal processing algorithms onthe audio samples, as will be described in detail presently, in order toprovide objective results. Thus, in the arrangement of FIG. 1, once atest connection is established, the setup should have the ability torecord voice signals from both sides of DUT 10 as well as to play voicesignals from both sides of DUT 10.

[0014] It may be desirable to have access to the data at points adjacentboth ends of DUT 10. However, for physical accessibility reasons, thismay sometimes be difficult to obtain. Additionally, it is ideal if thepath through device 20 does not affect the voice data. Again, this maynot always be the case. In particular, when device 20 is a PBX thesignal transmission path will have some acceptable amount of distortionor degradation. For these reasons, the tests must be able to allow forthat amount of degradation. In other words, the tests should notinterpret this amount of degradation as being unacceptable. The testdevice 44 and test host 42 may be used in order to capture the datawhile host 40 generates the data, and their roles may be reversed. Anyequipment that can perform this same function may replace these.

[0015] The tests carried out using the system and method make thefollowing assumptions. The foregoing acceptable amount of distortionprovided by PBX 20 may consist of DC offsets and AC gains with smallamounts of noise, filtering, or other disturbances. The PBX 20 must passdata through such that audio quality degradation is minimal. Thealgorithms should handle any distortion that the PBX 20 adds. Additionaldevices, such as voice processing boards, which are in or connected tothe test setup, pass the data through either unmodified, or with typical300 to 3000 Hz filtering.

[0016] Several alternatives are available for data collection. These maybe target dependent. One method is to use the computer file containingthe test signal exactly as is for the source. However, this may not beideal since the data may get changed before it is presented to the inputof DUT 10. Another method would be to capture the data at the input ofDUT 10, using a logic analyzer. A logic analyzer is a device designed tocapture patterns of data or voice signals. This method provides a gooddata source, however it may require custom setup of the logic analyzer,which is system dependent.

[0017] The sink or DUT output may be captured in ways much like thesource. One option is to capture the data through the PC, i.e., DUT Host40, and some sort of record operation. Again there could be somedifference between the captured data computer file and the DUT outputthat does not get accounted for. The advantage with this method is theease of implementation. The other methods of capturing directly on theDUT output are equivalent to the source capture methods of DUT input.

[0018] An example of a setup of these tests is shown in FIG. 2. Thissetup assumes a multi-port DUT 60 that is capable of making a connectionfrom one port to another. In this setup, an audio computer file may beplayed on the first port while being recorded on the second port. Thecomputer files may be compared directly on the DUT Host 62. The ports ofDUT 60 may be connected to DUT Host 62 and to PBX 64 as shown.

[0019] In cases where the setup of FIG. 2 cannot be used, thearrangement of FIGS. 3 and 4 provides an alternative. The device undertest (DUT) 70 has a data path 72 therethrough, PBX 74 is similar to PBX20 of FIG. 1 and PBX 64 of FIG. 2, and capture device 76 is a devicethat interfaces to PBX 74 and that can capture voice signals and putvoice signal patterns on the line. An example of device 76 is the voiceproduct commercially available from Voice Technologies Group division ofDialogic Corporation under the trademark iPOD. DUT host 80 may be apersonal computer or an embedded processor. Test host 82 may be aseparate host and likewise may be a personal computer or embeddedprocessor.

[0020]FIG. 3 illustrates a sink setup wherein capture device 76 may beused to record from DUT 70. In this setup, DUT host 80 may provide aplay signal function 84 and test host 82 may provide a record signalfunction 86. FIG. 4 illustrates a source set up wherein capture device76 is used to play into DUT 70. In this setup, DUT host 80 may provide arecord signal function 90 and test host 82 may provide a play signalfunction 92. In both setups, the computer files containing the voicesignals of interest may be collected on either DUT host 80 or test host82.

[0021] The setup of FIGS. 3 and 4 also may be used in situations wherethe data path passes through the transmit and receive of the deviceunder test and it is desired to isolate these paths in order to trackdown a problem.

[0022] The setup of FIGS. 3 and 4 requires connecting a capture devicelike device 76 to the same PBX as the device under test. For cases whereconnecting a capture device like device 76 to the same PBX as the DUT isnot possible, the setup of FIG. 5 provides an alternative. In thissetup, the two DUTs 100 and 102 may be used to perform the test. BothDUTs 100 and 102 are identical and are connected to PBX 104. IdenticalDUT hosts 106 and 108 are provided and connected to the DUTs 100 and102, respectively. DUT hosts 106 and 108 may be personal computers orembedded processors. DUT host 106 may provide a record signal function110 and DUT host 108 may provide a play signal function 112.

[0023] Evaluation of the voice data path requires that two samples ofaudio be obtained. A source signal should be played into one end of thedata path and then recorded at the other end. The formats of thecomputer files containing the signals should be similar and shouldrepresent what is seen at the endpoints of the DUT as best as possible.The source signal should be white gaussian noise (WGN) where the lengthof the computer file is much greater than the expected time betweenerrors. A WGN signal is chosen such that it can properly be aligned. Thestandard deviation of the signal should be set to ½ of the maximumsingle-ended range. Also, any random values that fall outside themaximum signal range should not clipped, but rather, regenerated untilthey fall inside the allowed range. Finally, the recording should bestarted before the source signal is played and stopped after the sourcesignal is stopped. It is also advantageous if the source computer filehits all possible values of data at least once.

[0024] This test may be used in order to capture errors of the followingtypes: random bit errors, data dependant bit errors, lost samples,repeated samples, synchronization problems (data shifted). This test maybe designed to catch errors that are difficult to capture because oftheir uncommon occurrence.

[0025] Once the signals are acquired, they may be presented to theanalysis system and method illustrated in FIG. 6 which contains severalalgorithms which will be described. Inputs to the system 120 shown inFIG. 6 may be the reference signal on path 122 and the captured signalon path 124. The system 120 may include an alignment stage 130, followedby an optional automatic gain control stage 132, then an optional filtercompensation stage 134, followed by a normalization stage 136 andfinally a comparison stage 138. The output of comparison stage 138either may be returned via a path 140 to alignment stage 130 for apurpose to be described, or it may be input to a stage 142 for utilizingthe result of the comparison.

[0026] Turning first to the alignment stage 130, it performs the firstoperation in the process which is to align the signals. Since a WGNsignal was used as a source, this operation may be performed by acorrelation. The correlation may be performed by taking a block of thesource signal and sliding it across the captured signal. The sourceblock that is used is the first block of the source file that containssignificant power. By significant power is meant power that exceeds ordistinguishes from background noise. This block may be correlatedagainst the captured signal and the result saved in an array. Inaddition, the energy of the source signal block may be computed as wellas the energy of the overlapping part of the captured signal. Next, thearray may be searched in order to find the maximum value. The index ofthis value may be used to adjust the captured signal such that it alignsproperly with the source block. An additional check is to make sure thatwhen the foregoing maximum value is squared, it is very close to theproduct of the two energies computed for the blocks of the correlationinput. This ensures that a true match has been found.

[0027] The foregoing is illustrated in further detail as follows. Thesource and captured signals are represented by the waveforms 150 and152, respectively, in FIG. 7. The correlation with the captured signalis performed by taking the first block of significant power of thesource signal. This is represented by the waveform 154 in FIG. 7. Thenext operation is taking the same block in time of the captured signal152. That block is represented by the rectangle 156 in FIG. 7. For eachsample in the source signal (s[i]) and captured signal (c[i]) thefollowing product sum is obtained:$y = {\sum\limits_{i = 1}^{n}{{s\lbrack i\rbrack} \cdot {c\lbrack i\rbrack}}}$

[0028] where N is the block length, i.e. the length of rectangle 156 inFIG. 7. The value of the product sum is stored in an array:

y[j]=y

[0029] The foregoing operations are repeated for a shifted version ofthe captured signal. Thus, the rectangle 156 would be shifted slightlyto the right as viewed in FIG. 7, but this is only for the capturedsignal, the same block being used. This would be represented by thefollowing:${y\lbrack j\rbrack} = {\sum\limits_{i = j}^{j + n - 1}{{s\lbrack i\rbrack} \cdot {c\lbrack i\rbrack}}}$

[0030] for all j (1 through the length of the source) which means:${y\lbrack 1\rbrack} = {\sum\limits_{i = 1}^{n}{{s\lbrack i\rbrack} \cdot {c\lbrack i\rbrack}}}$

${y\lbrack 2\rbrack} = {\sum\limits_{i = 1}^{n}{{s\lbrack i\rbrack} \cdot {c\left\lbrack {i + 1} \right\rbrack}}}$

[0031] and so on, i.e. y[1], y[2], y[3] etc. to the end of the filecontaining the signal.

[0032] The next operation is to use the array y[j] and find the maximumvalue where j is 1 through L, where L is the length of the signal (thelength of the file containing the signal). For example, if the maximumy[j] is at y[55] then the index of the maximum is 55. The location ofthe maximum is the location of the best fit.

[0033] Finally, taking the maximum index the source signal is shifted tothe left as viewed in FIG. 7 by the number of samples equal to themaximum index. This means the first x samples are discarded:

s[j] s[j+x]

[0034] where x is the maximum index and for all j from 1 to L where L isthe length of the file containing the signal. The signals now arealigned.

[0035] The next stage in system 120 is the automatic gain control (AGC)stage 132 wherein AGC may be optionally compensated for by removing afixed amount of data from the beginning of each computer file containingthe reference and captured signals. This allows the system time toadjust the gain, without substantially affecting the comparison.

[0036] An optional filter compensation may be performed in stage 134 ifthe audio path is known to contain filtering. The two signals are usedto feed a channel estimator, beginning at the alignment point. Thechannel estimator provides a model of the channel in the form of achannel filter. The resulting channel filter is then applied to thesource signal. Thus, as shown in FIG. 6, the filter component of stage134 is variable and controlled by the equalization component. Thiscompensation may cause some types of analog errors to go undetected.These analog errors are specifically defined as circuit errors thataffect the audio filtering. Errors in the analog portion of the digitalmodulation still will be detected.

[0037] Thus, the operation in stage 134 may be viewed as finding the“filter” which was applied to the captured signal so it can be appliedto the reference signal to provide the compensation. Since thetransmitted signal, i.e. the source or reference signal, may be filteredby the PBX 170 shown in FIG. 8A, the captured signal is then a filteredversion of the source:

c=f(s)

[0038] where f(s) is a filter applied to s.

[0039] Before a comparison of the reference (source) and capturedsignals is performed it may be desirable to filter the source signal sby the same filter. This is illustrated in FIG. 8B where f_(PBX)(s) is afiltering of the source (reference) signal s using an estimated filter.Stage 134 compensates this way, in order to ignore differences due to“allowed” filtering. The function f_(PBX)( ) is estimated using c, s andthe well known least means squares (LMS) algorithm.

[0040] After the signals have been aligned, they should be normalized inorder to remove any gain and bias that the system may have added.Normalization is performed in stage 136. The method is to use linearestimation in order to find the most likely system gain and bias. Theseestimates may be used to adjust the captured signal. This method assumesconstant gains and DC offsets over the entire length of the capturedsignal.

[0041] In particular, in order to compensate for a gain and DC offsetthe source signal is normalized by

Si _(new) =A·C+B

[0042] Optimal linearization shows that A and B are optimally given by:$A = {{\frac{\overset{\_}{SC} - \overset{\_}{SC}}{\overset{\_}{SS} - \overset{\_}{SS}}\quad B} = {\overset{\_}{C} - {A\overset{\_}{s}}}}$

[0043] where s=source signal and c=captured signal and$\overset{\_}{s} = {\sum\limits_{\frac{i = 1}{L}}^{L}{Si}}$

[0044] which is the mean of s

{overscore (c)}=mean of c

[0045] $\overset{\_}{sc} = {\sum\limits_{\frac{i = 1}{L}}^{L}{SiCi}}$

[0046] mean of the product${\overset{\_}{s}\quad \overset{\_}{c}} = {\left( {\sum\limits_{\frac{i = 1}{L}}^{L}{Si}} \right)\left( {\sum\limits_{\frac{i = 1}{L}}^{L}{Ci}} \right)}$

[0047] product of the mean

[0048] where L is the length over which the averaging occurs which inthis case is the length of the file containing the signal.

[0049] The signals may then be compared in stage 138 and if thedifference falls outside some tolerable range, an error may be declared.Once an error is found, a re-alignment may be performed via path 140.This may be accomplished by correlating a source block of length equalto the length where the differences are out of range, up to somemaximum. This block is correlated until a maximum correlation is found.Then the signal alignment is adjusted and the algorithm continues.Errors are flagged for later evaluation.

[0050] After all errors are found, they may be individually evaluated.All errors are evaluated by using the alignments before and after theerror. If the alignments are the same, it is called sample errors.Otherwise, the difference in alignment is reported as samples missed oradded.

[0051] The system and method of FIG. 6 may be in the form of softwarerun on a PC which can be in the DUT host. Alternatively, it may be runon a software layer in a section of the DUT which is not being tested.The software may be in the form of a program of instructions to evaluatethe data path, and the program may be embodied in a program storagedevice readable by a machine such as the foregoing PC or section of theDUT host and which instructions are executable by the machine.

[0052] While embodiments of the invention have been described in detail,that is done for the purpose of illustration, not limitation.

1. A data path evaluation system comprising: a) a device under testhaving a data path; b) a device to provide a signal path to the deviceunder test; c) a host arrangement operatively coupled to the signal pathproviding device to provide a test signal and to provide play and recordsignal functions; and d) the device under test being connected to thehost arrangement and to the signal path providing device so that thetest signal can be played and recorded in the data path for comparingthe played and recorded signals to evaluate the data path.
 2. The systemaccording to claim 1, wherein the host arrangement includes a first hostconnected to the device under test, a second host and a capture deviceoperatively connected to the second host and to the signal pathproviding device to provide a signal path therebetween.
 3. The systemaccording to claim 2, wherein a play signal function is provided in thefirst host and a record signal path is provided in the second host. 4.The system according to claim 2, wherein a record signal function isprovided in the first host and a play signal function is provided in thesecond host.
 5. The system according to claim 2, wherein each of thefirst and second hosts comprises a processor and the capture device is adevice which is identical to the device under test.
 6. The systemaccording to claim 2, wherein each of the first and second hostscomprises a processor and the capture device has the capability ofcapturing the test signal and placing signal patterns in the paththerethrough.
 7. The system according to claim 1, wherein the hostarrangement comprises a single host having a first port operativelyassociated with a signal play path and a second port operativelyassociated with a signal record path and wherein the device under testhas a first port connected to the first port of the host arrangement, asecond port connected to the second port of the host arrangement and apair of ports connected to the signal path defining device.
 8. Thesystem according to claim 5, wherein one of said first and second hostsincludes a play signal function and the other of said first and secondhosts includes a record signal function.
 9. The system according toclaim 1, wherein the data path of the device under test is a voicesignal path.
 10. The system according to claim 1, wherein the testsignal provided by the host arrangement contains white gaussian noise.11. The system according to claim 1, wherein the device to provide thesignal path is of the type to which the device under test will beoperatively coupled during use.
 12. The system according to claim 1,wherein the data path of the device under test has two ends and whereinthe test signal is played into one end of the data path and recordedfrom the other end.
 13. A method of evaluating the data path in a devicecomprising: a) sending a first signal through the data path; b)receiving a second signal from the data path; c) aligning the first andsecond signals; d) normalizing the aligned signals to remove gain andbias; and e) comparing the normalized signals to determine whether anydifference exists between the signals.
 14. The method according to claim13, further including realigning the compared first and second signalsif the difference exceeds the predetermined threshold.
 15. The methodaccording to claim 13, further including compensating for any AGC gainpresent in the aligned signals.
 16. The method according to claim 13,further including compensating the first and second signals for anyfiltering present in the data path.
 17. The method according to claim13, wherein the data path is a voice signal path.
 18. The methodaccording to claim 13, wherein the first signal contains white gaussiannoise.
 19. The method according to claim 13, wherein the first signal isa reference signal played into one end of the data path and the secondsignal is recorded from the other end of the data path to provide acaptured signal.
 20. The method according to claim 19, wherein thereference and captured signals are acquired prior to being aligned. 21.A program storage device readable by a machine, embodying a program ofinstructions executable by the machine to evaluate the data path in adevice, the instructions comprising: a) playing a reference signal intoone end of the data path and recording the signal from the other toprovide a captured signal; b) acquiring the reference and capturedsignals; c) aligning the reference and captured signals; d) normalizingthe aligned reference and captured signals to remove any gain and biasadded during the foregoing operations; and e) comparing the normalizedreference and captured signals to determine whether any differenceexists between the signals.
 22. The program storage device according toclaim 21, wherein the data path is a voice signal path.
 23. The programstorage device according to claim 21, wherein the reference signal iswhite gaussian noise.
 24. The program storage device according to claim21, further embodying the instructions of: a) compensating for any AGCgain present in the aligned reference and captured signals; and b)compensating the reference and captured signals for any filteringpresent in the data path.